WO 2005/069633 PCT/IB2005/050013 

1 

DISTRIBUTING CANDIDATE VECTORS 
BASED ON LOCAL MOTION COMPLEXITY 

The present invention generally relates to video processing applications, and more 
particularly, to systems and methods for distributing candidate vectors based on local 
motion complexity. 

For high quality video processing applications, motion estimation is typically used 
5 to perform a wide variety of beneficial functionalities, such as scan rate conversion, noise 
reduction, content encoding, and compression. In this regard, various motion estimation 
techniques have been developed to compensate for different characteristics of the video 
signal, different types of motion, and different objectives of the particular video processing 
application. 

10 These motion estimation techniques typically estimate motion for a block of pixels 

by evaluating a limited set of so-called candidate motion vectors. For each block of pixels, 
the motion estimation technique selects candidate vectors from a 3-D neighborhood by 
selecting from among the detected motion vectprs that are offset in the spatial and/or 
temporal domains. For example, candidate vectors may include one or more spatially offset 

1 5 motion vectors within the same frame and one or more temporally offset motion vectors 

* 

within a previous or subsequent frame. Additional "special" candidate vectors can also be 
used to take into account updated motion vectors or to compensate for zero motion, such as 
static text. In any event, the number of candidate vectors that are evaluated per block of 
pixels are typically fixed and predefined by the particular motion estimation technique. 
20 Every region, whether easy or difficult, is evaluated using the same number of candidate 

* 

vectors. 

Although fixing the number of candidate vectors simplifies the hardware and 
software implementation, these approaches can produce sub-optimal results. For example, 
in order to efficiently utilized limited computational resources and memory bandwidth, 

25 motion estimation should ideally adapt to the complexity of scene and the local content such 
that more emphasis is placed on picture regions that are difficult to estimate, while less 
emphasis is placed on regions that are easy to estimate (e.g. backgrounds with uniform 
motion). It has also been shown that evaluating many candidate vectors in picture regions 
with uniform (converged) motion can introduce annoying distortion into the video signal, 

30 and therefore do more harm than good. By using a fixed number of predefined candidate 
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vectors, existing motion estimation techniques can produce an inefficient or undesirable 
allocation of limited resources. 

Therefore, in light of the foregoing problems, there is a need for systems and 
methods that efficiently distribute candidate motion vectors based on local motion 
5 complexity. These systems and methods would preferably be used to improve the 
allocation of limited resources in video processing applications that employ a motion 
estimation technique. 

Embodiments of the present invention alleviate many of the foregoing problems by 

■ 

providing systems and method for distributing candidate vectors based on local motion 

10 complexity. In one embodiment, a picture frame is divided into a plurality of segments, 
where each segment includes a plurality of pixel blocks. The local motion complexity of 
each segment may then be measured by, for example, determining a Sum-bf- Absolute 
Differences (SAD) between pixel blocks of the frame and pixel blocks of an adjacent frame 
and summing the SAD values for the pixel blocks within each segment. The number of 

15 ■ candidate vectors assigned to each block may then be determined based on the measured 
local motion complexity. 

Other embodiments provide mechanisms for assigning the number of candidate 
: vectors to each pixel block.- In one embodiment, for example, a distribution function for 
distributing candidate vectors may be determined using a maximum, minimum and average 

20 SAD value for the segments within the picture frame. The distribution function may also 
use predetermined values for the maximum, minimum and average number of candidate* ■ 
vectors per block such that the distribution function achieves a desired distribution of 
candidate vectors. In this way, the distribution function will distribute a higher number of 
candidate vectors to pixel blocks within segments having a higher SAD value (high motion 

25 complexity) and a lower number of candidate vectors to pixel blocks within segments 

having a lower SAD value (low motion complexity). This non-uniform distribution of the 
number of candidate vectors can ensure that resources are re-directed to portions of the 
picture where such resources are needed. 

These and other features and advantages of the present invention will become more 

30 apparent to those skilled in the art from the following detailed description in conjunction 
with the appended drawings in which: 

Figure 1 illustrates an exemplary picture frame in accordance with one embodiment 
of the present invention; 
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Figure 2 illustrates an exemplary method in flowchart form for distributing 
candidate vectors in accordance with one embodiment of the present invention; and 

Figure 3 illustrates an exemplary platform that may be used in accordance with 
embodiments of the present invention. 
5 Embodiments of the present invention provide systems and methods for distributing 

candidate motion vectors based on local motion complexity. The following description is 
presented to enable a person skilled in the art to make and use the invention. Descriptions 
of specific applications are provided only as examples. Various modifications, substitutions 
and variations of the preferred embodiment will be readily apparent to those skilled in the 

10 art, and the generic principles defined herein may be applied to other embodiments and 

applications without departing from the scope of the invention. Thus, the present invention 
is not intended to be limited to the described and illustrated embodiments, and should be 
accorded the widest scope consistent with the principles and features disclosed herein. 
Embodiments of the present invention may be used to efficiently allocate limited 

15 computational resources to different portions of the picture frame without sacrificing 
motion estimation quality. In one embodiment, each picture frame is divided into a 
plurality of segments, where each segment includes a plurality of pixel blocks. The number 
of candidate vectors evaluated for pixel blocks within each segment may then be 
determined based on an estimation of the relative motion complexity of each corresponding 

20 segment. As such, the pixel blocks within the same segment are allocated the same number 
of candidate vectors, but pixel blocks within different segments could be allocated a. 
different number of candidate vectors depending on the relative motion complexity of the 
different segments. This approach creates a non-uniform distribution of candidate motion 
vectors (and associated computational resources used in their evaluation) for different 

25 groups of pixel blocks based on the relative motion complexity within each group. 

In order to estimate the relative motion complexity of each segment, the Sum-of- 
Absolute-Differences (SAD) between pixels of a frame and corresponding pixels of an 
adjacent frame may be calculated, where the corresponding pixels of the adjacent frame 
may be spatially aligned or spatially offset in accordance with a candidate motion vector. 

30 Because the SAD value in this case calculates the sum of the absolute value of differences 
of pixels between blocks of adjacent frames, the SAD value of each segment provides the 
mismatch error of the blocks within the segment, and therefore, can be used as an indirect 
measure of the motion complexity of each segment If a segment has a high SAD value, 
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then the segment likely has high motion complexity, and therefore the pixel blocks within 
the segment may benefit from a greater number of candidate motion vectors. If a segment 
has a low SAD value, then the segment likely has low motion complexity, and therefore, it 
may be more efficient to allocate a lower number of candidate vectors to the pixel blocks 
5 within that segment. By distributing a higher or lower number of candidate vectors based 
on the relative motion complexity of each segment, limited computational resources can be 
non-uniformly distributed over different portions of the picture based on relative need. 

Referring to Figure 1 , an exemplary picture frame in accordance with one 
embodiment of the present invention is illustrated generally at 100. As illustrated, the 

10 exemplary frame 1 10 is divided into twenty-four picture segments 120, where each segment 
120 includes sixteen 8x8 pixel blocks 130. For purposes of clarity, the pixel blocks 130 are 
only illustrated for the first segment 120. It is understood that the exemplary frame of 
Figure 1 is provided for illustration purposes only and that other subdivisions of the picture 
frame are contemplated and embraced by the present invention. 
'15 In order to estimate the motion complexity for each segment 120, a SAD value is ■•: 

calculated between pixels blocks 1 30 of one frame 1 1 0a and corresponding pixel blocks 130 
of an adjacent frame 1 10b. In this context, it is understood that the corresponding pixel 
■ blocks 130 of the adjacent frame 1 10b could be spatially aligned with the pixel blocks 130 
. of the current frame 1 10a. Alternatively, the corresponding pixel blocks 130 of the adjacent 

20 frame 1 10b could be spatially offset with respect to pixel blocks 130 in the current frame 

1 10a based on an estimated motion vector that is determined using a limited set of candidate 
motion vectors. Accordingly, for embodiments of the present invention, the corresponding 
pixel blocks 130 of adjacent frames 1 10b could be spatially aligned or spatially offset with 
respect to the pixel blocks of the current frame 1 10a. 

25 Since each segment 120 includes a plurality of pixel blocks 130, the SAD values for 

each target segment 120 can be determined by summing the SAD values for the individual 
pixel blocks 130 within the target segment 120. As such, the SAD value for each segment 
120 provides an estimate of mismatch error between the pixel blocks 130 within each 
segment 120, and therefore, an estimate of the local motion complexity of each segment 

30 120. 

Once the SAD value for each segment 120 has been determined, a distribution 
function can be used to determine the number of candidate vectors to be evaluated for pixel 
blocks 130 within each segment 120 to produce a desired distribution of candidate vectors. 
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For example, suppose that there are sufficient resources to evaluate on average N_av 
number of candidates motion vectors for all the blocks 130 within a frame 1 10. Further 
suppose that it is desired to limit the number of candidate vectors to be evaluated to Njmax 
and N_min for the maximum and minimum number of candidates per block 130, 
5 respectively. A distribution function for candidate vectors may then be determined by 
applying the following rules: (1) evaluate Njnin number of candidate vectors for pixel 
blocks 130 within segments that have the lowest match error (lowest SAD); (2) evaluate 
N_max number of candidate vectors for pixel blocks 130 within segments that have the 
highest match-error (highest SAD); and (3) evaluate on average N_av number of candidate 
10 vectors per block for the blocks 130 within segments that have the average match-error. 
Based on these rules, the following exemplary equations would apply: 

Njnin = a + b*SAD_min + c*SAD_min*SAD_min (1) 
Nmax= a + b*SADjnax + c*SADjmax*SAD max (2) 
N_av = a + b*SAD_av + c*SAD_av*SAD_av (3) 

15 In equations (1) - (3), SAD_max, SADjmin and SAD_av correspond to the 

maximum, minimum and average SAD values, respectively, of all segments within the . 
targeted frame. Furthermore, Njnax, Njnin and N_av can be predetermined values for the 
maximum, minimum and average number of candidate vectors, respectively, that produce a 
desired distribution of candidate vectors among the pixel blocks. In a preferred 

20 embodiment, N_max is 9; N_min is 4; and N_av is 6. 

If equations (1) - (3) above are solved simultaneously to determine the values of a, b 
and c, the following equation can be used to determine the number of candidate vectors to 
be evaluated within each segment based on the segment's corresponding SAD value: 
N_candi = a + b*SADi + c*SADi*SADi (4) 

25 ,where N_candj is the number of candidate vector to be evaluated for pixel blocks within the 
i-th segment, and SADj is the SAD value for the i-th segment. By using equation (4) to 
determine the number of candidate vectors, the available number of motion vectors to 
evaluate can be distributed over the various pixel blocks according to the local motion 
complexity of each segment, i.e. the resources are spend at the locations where they are 

30 needed. Hie distribution function of equation (4) also allocates the number of candidate 
vectors such that distribution of candidate vectors conforms with the desired distribution 
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(e.g., if SADj = SADmax, then N_candi = Njmax; if SADj = SADmin, then N_candj = 
Njnin; and so forth). 

In comparison to schemes that use a fixed number of motion candidates per pixel 
block, the presented scheme can help to: (1) provide an improved motion estimation quality 
5 with the same total number of candidate vectors, and/or (2) provide a similar motion 

estimation quality with a fewer number of candidate vectors. For those skilled in the art, it 
is obvious that other sets of equations can be used and a different expression can be 
resolved to arrive at a (different) distribution function. The presented set of distribution 
equations is just an example. Moreover, different or multiple measurements of local motion 

10 complexity can be taken into account. For example, instead of using SAD values, 

embodiments of the present invention could use another estimate of mismatch error, such as 
Mean-Squared Error (MSE), without departing from the principles of the present invention. 

Referring to Figure 2, an exemplary method in flowchart form for distributing 
candidate vectors in accordance with one embodiment of the present invention is illustrated 

1 5 generally at 200. As illustrated, the exemplary method measures the local motion 

complexity at step 210 by, for example, measuring SAD or MSE values for a plurality of 
pixel blocks and summing the SAD or MSE values for a group of pixel blocks. A 
distribution function for determining the number of candidate vectors assigned to each pixel 
block is then determined at step 220. This process may involve developing a set of 

20 distribution equations, such as equations (1) - (4) above, that define a desired distribution of 
candidate vectors based on the measured local motion complexity. These equations may 
include one or more anchor values, such a maximum, minimum or average value for the 
measured motion complexity. 

At step 230, the number of candidate vectors to be evaluated for each pixel block is 

25 determined using, for example, the distribution function of equation (4) and the measured 
local complexity of the pixel blocks. Motion estimation for the applicable video frame may 
then be performed at step 240 using the number of candidate vectors assigned to each pixel 
block. It should be noted that the motion estimation technique should take into account the 
fact that a different number of candidate vectors may be assigned to different pixel blocks. 

30 For example, if the candidate vectors.that could be evaluated include one or more spatial 
vectors, one or more temporal vectors, an update (modified motion) vector, a global motion 
vector, and a zero motion vector, the motion estimation technique may assign a priority 
order to each type of candidate vector. If the number of candidate vectors assigned to a 
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pixel block is insufficient to evaluate all the possible candidate vectors, the motion 
estimation technique can then select from among the types of candidate vectors based on the 
vectors' relative priority. In this way, the most relevant motion vectors would be evaluated 
for pixel blocks that are assigned a relatively small number of candidate vectors. 
5 Furthermore, if the number of candidate vectors are selected such that a smaller number of 
candidate vectors are assigned to pixel blocks with relatively simple motion complexity, 
this limited set of motion vectors may be sufficient to compensate for the motion involved 
without sacrificing motion estimation quality. 

Referring to Figure 3, an exemplary platform that may be used in accordance with 

10 embodiments of the present invention is illustrated generally at 300. As illustrated, the 
exemplary platform includes a microprocessor 310 operably coupled to a memory system 
320 via a system bus 340. The memory system may comprise a random access memory, a 
hard drive, floppy drive, a compact disk, or other computer readable medium, that stores 
computer instructions for a motion estimation module 350 and video processing. application 

15 150. The exemplary system also includes an I/O interface 330 that is coupled to the. 

microprocessor 310 and the memory system 320 in order to enable the system to input and 
output video signals. 

In operation, the video processing application 360 uses the motion estimation 
module 350 to process video signals received from the I/O interface 330 and to output the 

20 processed video signal to the I/O interface 330. If the motion estimation module 350 is 

implemented in accordance with the principles described above, the exemplary system may 
be configured to efficiently utilize the limited processing resources of the microprocessor 
310 and the limited memory bandwidth of the memory system 350 without sacrificing 
motion estimation quality. Furthermore, if the motion estimation module 350 and/or video 

25 processing application 360 are implemented in hardware or combinations of hardware and 
software (instead of the software only implementation illustrated in Figure 3), other 
embodiments of the exemplary system would provide the system designer flexibility in 
designing the optimal configuration. 

While the present invention has been described with reference to exemplary 

3.0 embodiments, it will be readily apparent to those skilled in the art that the invention is not 
limited to the disclosed and illustrated embodiments but, on the contrary, is intended to 
cover numerous other modifications, substitutions and variations and broad equivalent 
arrangements that are included within the scope of the following claims. 



